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Abstract 

Background: Type 2 Diabetes Mellitus (DM) is a common chronic disease associated with multiple clinical 
complications. Management guidelines have been established which recommend a risk-stratified approach to 
managing these patients in primary care. This study aims to evaluate the quality of care (QOC) and effectiveness of 
a multi-disciplinary risk assessment and management programme (RAMP) for type 2 diabetic patients attending 
government-funded primary care clinics in Hong Kong. The evaluation will be conducted using a structured and 
comprehensive evidence-based evaluation framework. 

Method/design: For evaluation of the quality of care, a longitudinal study will be conducted using the Action 
Learning and Audit Spiral methodologies to measure whether the pre-set target standards for criteria related to the 
structure and process of care are achieved. Each participating clinic will be invited to complete a Structure of Care 
Questionnaire evaluating pre-defined indicators which reflect the setting in which care is delivered, while process of 
care will be evaluated against the pre-defined indicators in the evaluation framework. 

Effectiveness of the programme will be evaluated in terms of clinical outcomes, service utilization outcomes, and 
patient-reported outcomes. A cohort study will be conducted on all eligible diabetic patients who have enrolled 
into RAMP for more than one year to compare their clinical and public service utilization outcomes of RAMP 
participants and non-participants. Clinical outcome measures will include HbA1c, blood pressure (both systolic and 
diastolic), lipids (low-density lipoprotein cholesterol) and future cardiovascular diseases risk prediction; and public 
health service utilization rate will include general and specialist outpatient, emergency department attendances, 
and hospital admissions annually within 5 years. For patient-reported outcomes, a total of 550 participants and 
another 550 non-participants will be followed by telephone to monitor quality of life, patient enablement, global 
rating of change in health and private health service utilization at baseline, 6, 12, 36 and 60 months. 

Discussion: The quality of care and effectiveness of the RAMP in enhancing the health for patients with type 2 
diabetes will be determined. Possible areas for quality enhancement will be identified and standards of good 
practice can be established. The information will be useful in guiding service planning and policy decision making. 

Keywords: Type 2 diabetes mellitus, Quality of life, Primary care, Prevention, Primary health care, Management 
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* Correspondence: cfsc@hku.hk 

department of Family Medicine and Primary Care, The University of Hong 
Kong, 3/F Ap Lei Chau Clinic, 161 Main Street, Ap Lei Chau, Hong Kong 
Full list of author information is available at the end of the article 

O© 2012 Fung et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative 
BiolVlGCl C6ntTcll Commons Attribution License (http://creativecommons.Org/licenses/by/2.0), which permits unrestricted use, distribution, and 
reproduction in any medium, provided the original work is properly cited. 



Fung et al. BMC Family Practice 201 2, 1 3:1 1 6 
http://www.biomedcentral.eom/1 471-2296/1 3/1 1 6 



Page 2 of 9 



Background 

Delivery of public-sector primary care in Hong Kong 
occurs through government-funded General Out- Patient 
Clinics (GOPC) managed by the Hospital Authority 
(HA). There are currently 74 clinics divided into seven 
geographical districts and referred to as HA clusters 
servicing the population of Hong Kong requiring 
government-subsidized health care. These primary care 
clinics provided 4,979,754 general outpatient atten- 
dances in 2010-2011 [1] and the number of atten- 
dances was estimated to increase further in the coming 
years. The majority of patients are elderly, of lower 
socio-economic status or have chronic diseases which 
require regular monitoring or medication such as 
hypertension and diabetes mellitus. 

Type 2 diabetes mellitus (DM) is a major cause of 
morbidity and was the ninth commonest cause of death 
in Hong Kong in 2008 [2]. There are approximately 
190,000 patients receiving care for DM in the GOPC 
according to data from HA. Until recently however, 
there have not been any formal standardized guidelines 
or protocols regarding the delivery of care for diabetic 
patients within the public primary care setting. In August 
2009, the HA introduced a multi-disciplinary Risk As- 
sessment and Management Programme (RAMP) to im- 
prove the quality of care for patients receiving diabetic 
care in the GOPCs. The RAMP utilizes a standardized 
protocol consisting of a workflow of checking of relevant 
clinical parameters including HbAlc, blood pressure 
(BP), low density lipoprotein-cholesterol (LDL-C), and 
an agreed risk assessment criteria for risk level stratifica- 
tion, with different management options assigned to 
patients of different risk levels and with different needs. 
Patients with DM who are independent in their activities 
of daily living and being followed up at regularly at the 
GOPCs are eligible to enter the RAMP. All enrolled 
patients undergo a comprehensive risk assessment and 
screening for diabetes-related complications, and are 
then assigned to receive appropriate interventions and 
education provided by a team of multi-disciplinary 
healthcare professionals according to their stratified risk 
level. Low risk patients continue with the usual GOPC 
follow up, medium risk patients are given additional 
intervention by an advanced practice nurse (APN), and 
high risk/very high risk patients are given additional 
intervention by an APN and an associate consultant, 
who is a specialist family physician. The RAMP is 
repeated once at least every one to two years for all 
patients who are enrolled. 

A review of the international literature on risk stratifi- 
cation of diabetic patients supports the benefit of identi- 
fying high-risk patients through the clinical information 
system [3]. Risk factors can be identified based on the level 
of glycaemic control and/or presence of complications 



where improvement in HbAlc, BP, and LDL-C can be 
achieved through more intensive interventions [3,4]. 
Moreover, health service utilization such as Accidents and 
Emergency (A&E) department attendance, consultations, 
and hospital admissions can also be reduced [5,6]. A num- 
ber of non-Asian countries have already successfully 
adopted such stratified models for chronic diabetic care 
including the United Kingdom, Australia, New Zealand 
and Canada [7-11]. 

In Asia, the Joint Asia Diabetes Evaluation (JADE) 
Program, incorporating a comprehensive risk engine, 
care protocol, clinical decision and self-management 
support has recently been developed to improve ambula- 
tory diabetes care [12]. Although the JADE was deve- 
loped based on data from patients receiving secondary 
or tertiary care, it forms the base for the stratification of 
local diabetic patients into very high risk, high risk, 
medium risk and low risk in the RAMP at primary care 
level. Apart from stratifying diabetic patients into differ- 
ent risk levels, equations integrating different patients 
clinical parameters have been formulated to predict the 
diabetic patients 5-year risk of coronary heart disease, 
stroke, end-stage renal disease, and all-cause mortality 
[12-15]. 

The goal of care for patients with DM is to prevent 
DM-related complications such as cardiovascular dis- 
eases and renal failure. Cardiovascular risk prediction 
rules have been developed, mostly based on the Framing- 
ham Study [16], but these may not be applicable to the 
Asian or Chinese population [17,18]. Studies have shown 
that the original Framingham functions might over- 
estimate the risk of coronary heart disease (CHD) in Chin- 
ese adults [18]. The Joint Asia Diabetes Evaluation (JADE) 
equations have been formulated and tested on local data 
from the Hong Kong Diabetes Registry showing good va- 
lidity and discriminating power. 

With the threat of aging population and the foreseeable 
increase in demand of public primary healthcare ser- 
vice on providing care to patients with chronic dis- 
eases, programmes that can be proved to be effective 
and provide a good quality of care are one of the 
solutions to deal with the complexity and demand of 
the health needs. Since DM is a common chronic 
condition with diverse complications in our locality 
as well as worldwide, we would like to evaluate the 
QOC and the effectiveness of the RAMP to prove if 
such kind of approach of chronic disease manage- 
ment works. 

The evaluation of QOC and effectiveness is an essen- 
tial step in assessing a chronic disease management 
programme on whether the intended care is provided 
and the expected health benefit is achieved. The infor- 
mation will influence future policy and service planning 
related to healthcare. 
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Aims and objectives 

The aim of this study is to evaluate the quality of care 
(QOC) and effectiveness of a multi-disciplinary risk as- 
sessment and management programme (RAMP) for type 
2 diabetic patients attending government-funded pri- 
mary care clinics in Hong Kong. The evaluation will be 
conducted using a structured and comprehensive 
evidence-based evaluation framework. 
The objectives of the study are: 

1) To review and identify the structure, process and 
outcome indicators of quality of care; 

2) To identify the criterion and set the target standard 
for each indicator; 

3) To compare the observed standards against the 
target standards; 

4) To identify any on-site problems related to 
implantation of the RAMP; 

5) To provide feedback on quality of care of RAMP; 

6) To identify possible areas for improvement; 

7) To give recommendations for enhancement of 
service delivery 

The following hypotheses will be tested: 

1) The structure and process criteria of care of different 
aspects should be achieved up to standards in all 
participating clinics; 

2) A higher proportion of patients should have 
achieved the outcome targets for HbAlc, blood 
pressure, and LDL-C after the RAMP; 

3) Patients participated in the RAMP should have 
better clinical outcomes than non-participants 
(control) managed by usual care; 

4) Patients participated in the RAMP should not have 
higher health service utilization rates than non- 
participants (control) managed by usual care; 

5) In longer terms, RAMP will lower the complication 
rate and cardiovascular risk level of diabetic patients. 

Methods/design 

Evaluation of quality of care 

A longitudinal study using the Action Learning [19] and 
Audit Spiral methodologies [20] will be used to carry 
out a systematic analysis of the quality of care, and to 
identify areas for enhancement in a multi-disciplinary 
approach to chronic disease management. Audit is the 
best available tool to evaluate whether patients are re- 
ceiving the best quality of care. It provides information 
to the service provider on how to improve the quality of 
health care delivery. It also reviews the procedures for 
assessment of patients, service delivery, outcomes and 
resources allocation. The audit can be regarded as a 
spiral systematic process, with the ultimate goal of 



improving the quality of care. The audit consists of dif- 
ferent cycles and as the audit cycle continues, each cycle 
aims at a higher level of quality of care. 

Development of the evaluation framework 

Donabedian s taxonomy of quality of care on structure, 
processes and outcomes will be used as the evaluation 
framework [21]. A QOC framework (Additional file 1) 
was developed by an iterative process and reconciliation 
between the investigators and the programme providers 
(Figure 1). It started with the intensive literature review 
on DM so that key elements of management of DM at 
primary care level were grouped. Through iterative dis- 
cussions with the working group (WG) of the health 
care providers (HA) and the statistical team of the HA 
on what aspects needed to be evaluated and what data 
are retrievable from the computer system, the evaluation 
framework gradually came into shape. This final eva- 
luation framework listed out the indicators of the struc- 
ture (staff, facilities, organization, and management), 
process (what, when and how care is delivered), and out- 
comes (clinical outcomes, service utilization and patient 
reported outcomes) with the required criteria and target 
standard of care to be achieved. 

Evaluation of the effectiveness of RAMP 

Effectiveness of the programme will be evaluated in 
terms of clinical outcomes, service utilization outcomes, 
and patient-reported outcomes. A cohort study will be 
conducted on all eligible diabetic patients who have en- 
rolled into RAMP for more than one year to compare 
their clinical and public service utilization outcomes of 
RAMP participants and non-participants. Clinical out- 
come measures will include HbAlc, blood pressure 
(both systolic and diastolic), lipids (low-density lipopro- 
tein cholesterol); and the 5-yr and 10-yr cardiovascular 
risk or coronary heart disease risk (by using appropriate 
risk prediction equations). Public health service 
utilization rate will include general and specialist out- 
patient attendances, emergency department attendances, 
and hospital admissions. This cohort of patients will be 
followed up regularly and evaluated at assigned time- 
points, i.e. 6, 12, 36 and 60 months after the start of the 
study. For evaluation of the patient reported outcomes 
(PRO), 550 diabetic patients enrolled into RAMP and 
550 diabetic non-RAMP patients will be recruited for 
telephone interview at the same time interval for their 
PEI, GRS, and private health service utilization rate. 

Subjects 

All diabetic patients who have been enrolled into the 
RAMP will be included in the evaluation on process of 
care for each audit cycle. All enrolled subjects who have 
been recruited for more than 12 months in the 
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Step 1 : HKU evaluation team 
drafted generic quality of care 
indicator framework 



Step 3: HKU evaluation team 
refined programme specific 
indicators, criteria & standards 



Step 5: Programme WG and HKU 
evaluation team reconciliation on 
indictors, criteria, standards and 
operational definitions 




Step 2: Programme WG defined 
indicators and criteria 



Step 4: Programme WG 
specified detailed setting of 
indicators, criteria & standards 



Step 6: Programme WG reviewed 
and finalized the framework that 
is evidence-based and practical 



Figure 1 An iterative process and reconciliation between the HKU evaluation team and the programme working group (WG). 



programme will be included in the evaluation on the 
clinical outcomes of care for each audit cycle. 

To identify the difference in programme perfor- 
mance and impact between the two groups (RAMP 
participants and non-participants), a cohort study ap- 
proach would be adopted. The cohort includes all eli- 
gible diabetic patients under care of the public 
primary care outpatient clinics at 1 Sep 2009. The co- 
hort would be evaluated annually on their clinical and 
service utilization outcomes from September 2009 to 
September 2015. 

550 diabetic patients who have enrolled into the 
RAMP and another 550 diabetic patients who have not 
will be invited in person from the GOPCs by trained re- 
search assistants at the beginning of each audit cycle to 
take part in a telephone survey on PRO. All subjects 
who have signed the written consent to the telephone 
survey will be interviewed by telephone within four 
weeks, at 6, 12, 36 and 60 months. 

Sample size calculation 

The sample size for the evaluation of quality of life 
change after RAMP is estimated to detect a minimally 
clinically important difference (MCID) in health-related 
quality of life (HRQOL) studies that is equivalent to 
Cohens small effect size of 0.3 [22]. A sample size of 
350 patients in total (175 RAMP participants and 175 
non-participants) is needed in order to have 80% power 
and 95% confidence interval by independent T-test [23]. 
Therefore taking into consideration these factors and 
allowing for a 25% dropout rate on each follow up, a 
total of 1100 patients (550 RAMP participants and 550 
non-participants) need to be recruited. 

Data collection 

Evaluation on structure and process 

The coordinator of each of the seven clusters will be 
asked to complete a Structure of Care Questionnaire 



(Additional file 2). Anonymised data will be retrieved 
from the Hospital Authority's patient information system 
to determine the patient recruitment rate, enrolment 
rate, risk level stratification, attendance at RAMP clinic, 
compliance with assessment as per protocol, number of 
investigations and referral rate. Information on patient 
and provider characteristics will also be collected as in- 
dependent variables. The doctor-in-charge of the RAMP 
of each participating clinic will also complete the struc- 
ture of care questionnaire (Additional file 2). We will 
ask them to submit a list of the staff and facilities desig- 
nated for the programme, and a description of the 
programme objectives and protocol, and we will carry 
out site visits to cross-validate the data. 



Evaluation on outcomes of care 

Anonymised data on the relevant clinical outcomes, an- 
nual attendance rates for GOPC, specialist outpatient 
clinics (SOPC), A&E department and hospital admis- 
sions at baseline, 12, 24, 36 and 48 months will be 
retrieved from the Hospital Authority's information sys- 
tem. The data will be extracted using specific data col- 
lection form. Data needed to calculate the future 
cardiovascular risk prediction (e.g. urine albumin to cre- 
atinine ratio) will be extracted from HAs information 
system. 

The 550 RAMP participants and 550 non-participants 
who have agreed to the telephone follow up will be con- 
tacted by a trained research assistant who will obtain 
written consent from each patient. Each patient will then 
answer the Chinese (Hong Kong) Short Form- 12 version 
2 (SF-12v2) Health Survey within four weeks from enrol- 
ment and a repeat survey at 6, 12, 36 and 60 months to- 
gether with an assessment on patient enablement and 
global rating of change in health condition and a struc- 
tured questionnaire on private health service utilization 
rates. 
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Three audit and feedback cycles 

An interim analysis of the standards of care, particularly 
on the structure and process of care was carried out 
6 months after the start date of the evaluation study to 
identify major problems of programme implementation. 
These results were fed back and discussed with the 
programme team of HA to identify areas of deficiency 
and quality enhancement strategies. 

Three evaluation (audit) cycles on the standards of 
care will be carried out at 15, 30 and 45 months from 
the start of the study to evaluate whether the set stan- 
dards of care on structure, process and outcomes have 
been achieved (Figure 2). The first evaluation cycle is to 
identify gaps between practice and intended targets that 
may require changes in the structure and process of 
care. The second evaluation cycle is to determine the 
standards that are achievable after the programme has 
been established. The third evaluation cycle is to assess 
the sustainability of the standards of care. The results of 
each evaluation cycle on the performance of the 
programme will be fed back and discussed with the 
programme team of the Hospital Authority to identify 
any need for changes in quality criteria and standards 
and corresponding quality enhancement strategies to be 
implemented for RAMP. 

Outcome measures 
Primary 

1. The proportion of clinics that have satisfied the 
target of each of the structure criterion. 

2. The number and proportion of diabetic patients who 
have completed RAMP. 

3. The proportion of patients who have complied with 
the target of each of the process of care criterion. 



4. The proportion of patients who have achieved the 
target of each of the outcome of care criterion, 
mainly the control of DM, as reflected by HbAlc. 

5. DM-related complication rates, e.g. coronary heart 
disease, stroke, all-cause mortality, etc. 

6. The predicted 5-year / 10-year cardiovascular risk, by 
using the risk prediction equations. 

Secondary 

1. The change in risk level of diabetic patients after 
RAMP 

2. Other clinical outcomes, including SBP, DBP, LDL-C, 
and body mass index. 

3. PRO measured by the change in SF-12v2 scores, the 
PEI and GRS scores at 6, 12, 36 and 60 months. 

4. Service utilization outcomes measured by GOPC and 
SOPC attendance rates, A&E and hospital admission 
rates in the past 12 months. 

Study instruments 

Study instruments will be used in evaluating the struc- 
ture of care and the PRO. Evaluation of the process of 
care and outcomes do not involve use of study instru- 
ments as the necessary data is to be retrieved from the 
HA through the statistical team. 

Structure of care questionnaire 

Structure of care questionnaire will be sent to the cluster 
co-ordinators and clinic doctors in-charge of the RAMP 
and require their input. The questionnaire covers ques- 
tions on resources spent on RAMP, e.g. whether there is 
enough space for the programme, whether there is a 
data sharing platform within the programme, etc. 



Third evaluation on 
quality of care & 
effectiveness of 
programme 



Second evaluation on 
the standards of 
structure, process and 
outcome of care 



First evaluation on 
the standards of 
structure, process and 
outcome of care 



Feedback of results to 
Programme WG; 
review criteria & 
standards; action plan 
for improvement 



Feedback of results to 
Programme WG; 
review criteria & 
standards; action plan 
for improvement 

Programme WG and 
HKU evaluation 
team set target 
criteria & standards 



Figure 2 The audit spiral with three evaluation (audit) cycles. 
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Three study instruments are used in the evaluation of 
the PRO, namely the short form- 12 version 2 Health 
Survey, the Patient Enablement Instrument, and the 
Global Rating Scale. 

A. The short form- 12 version 2 (SF-12v2) Health 
survey 

The Chinese (Hong Kong) SF-12v2 Health Survey 
will be used to measure HRQOL. It has been 
validated and normed on the general Chinese 
population in Hong Kong [24]. It measures eight 
domains of HRQOL on physical functioning, role 
physical, bodily pain, general health, vitality, social 
functioning, role emotional and mental health on a 
scale range from 0 to 100. A higher score indicates 
better HRQOL. The eight domain scores can be 
summarized into two summary scores, the physical 
(PCS) and mental (MCS) component summary. 

B. The Patient Enablement Instrument (PEI) 

The PEI is a measure of patients enablement in 
coping with the illness and life [25] . It has 6 items 
each rated on a 3-point (0, 1, and 2) scale. The 
summation of the item scores gives the PEI score 
with a higher score indicating better enablement. 
The PEI has been translated into Chinese and shown 
to be valid and reliable in the general Chinese 
population [26]. 

C. The Global Rating Scale (GRS) 

The GRS is adapted from those used in studies by 
Jaeschke and Osoba et al. [27]. It assesses the 
subjects global perception of any change in the 
overall health condition on a 7-point scale (-3, -2, 
-1, 0, 1, 2, and 3) over the last six months. 

Data analysis 

Descriptive statistics on standard of care will be calcu- 
lated by the percentage of clinics meeting the structure 
criteria, and percentage of subjects enrolled, attended, 
completed the programme, receiving criterion process of 
care, investigations and referral per protocol, and per- 
centage of subjects achieving the criterion outcomes of 
care, in each audit cycle, and at the end of five years. 
Moreover, our primary analysis will use cardiovascular 
disease (CVD) risk-prediction algorithm according to 
the Framingham Heart Study [16]. This equation pre- 
dicts the 10-year risk of CVD events which is defined as 
a composite of CHD (coronary death, myocardial infarc- 
tion, coronary insufficiency, and angina), cerebrovascular 
events (including ischemic stroke, haemorrhagic stroke, 
and transient ischemic attack), peripheral artery disease 
(intermittent claudication), and heart failure. The equa- 
tion is applicable to patients aged 30-74 years. As a 
means of cross-validation and comparison, the predicted 
5 -year cardiovascular risk will also be estimated by the 



locally validated JADE equations [12-15]. In addition, 
within-subject improvement in outcomes from baseline 
at 12, 24, 36 and 48 months will be analysed by paired t- 
test for continuous outcomes and McNemar test for 
binary outcomes. Furthermore, differences in outcomes 
between RAMP participants and non-participants, be- 
tween audit cycles will be tested by analysis of variance 
(ANOVA) for continuous outcomes and Chi-square test 
for categorical outcomes. We will also compare the 
baseline frequency of attendance between RAMP partici- 
pants and non-participants as a proxy measure of their 
motivation to care. Self-selection biases will be examined 
over inter-subject and inter-group differences by analysis 
of covariance. Adjustment for multiplicity where appro- 
priate will be made by Bonferroni adjustment. At last, 
multiple regressions will be used to identify factors that 
are associated with quality of care or effectiveness of the 
RAMP. 

Ethics Approvals 

This study has received ethics approval from the Institu- 
tional Review Board of the University of Hong Kong/ 
Hospital Authority Hong Kong West (UW 10-369), 
Hong Kong East (HKEC-2010-093), Kowloon East and 
Kowloon Central (KC/KE-10-0210/ER-3), Kowloon West 
(KW/EX/10-317 (34-04)), New Territories East (CRE- 
2010.543), and New Territories West clusters (NTWC/ 
CREC/1091/12). 

Discussion 

Quality assurance exercise on such a scale has never 
been conducted in our settings, and only few have been 
conducted in Asia. Careful attention is required to facili- 
tate coordination between the various parties. This is the 
first territory wide study to evaluate the quality of care 
of a complex systematic intervention for diabetic 
patients in Hong Kong. The results can assure the qual- 
ity and determine the effectiveness of this multi- 
disciplinary programme in improving the health and 
health-related outcomes for patients with DM. Possible 
areas for quality enhancement will be identified and 
standards of good practice can be established. Action re- 
search requires the engagement of key stakeholders' 
right from the beginning of protocol development and 
readiness to make changes in response to new issues 
that merged during the study process. Close collabor- 
ation with the programme administrator and providers 
are vital so that the results are valid and that recommen- 
dations made are feasible and acceptable. 

Coordination of stakeholder collaboration and 
communication 

As the study involved health policy decision makers, 
Hospital Authority administrators and other staff from 
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the Hospital Authority Head Office as well as Chiefs -of- 
service from seven regional clusters in addition to nu- 
merous frontline clinical staff, adequate and open chan- 
nels for communication is essential for the smooth 
facilitation of the project. A senior project coordinator 
and single point of call representing the academic inves- 
tigators and Hospital Authority teams helps to facilitate 
and streamline communication and arrangement of ac- 
tivities such as meetings, site visits, subject recruitment 
(for PRO) and monitoring of data collection. 

Regular planning and feedback meetings are required 
to ensure all parties receive timely updates to the status 
of the evaluation and sharing of interim findings. The 
use of an iterative process is useful to facilitate stakeholder 
participation, however, careful planning is required to 
allow for efficient and effective two-way communica- 
tion. Modifications to the evaluation framework, data 
collection and dissemination of results require endorse- 
ment by the decision-makers and the establishment of 
working groups is an effective way to manage the ad- 
ministrative and logistical challenges encountered when 
dealing with multiple parties. 

Feedback and planning meetings 

Continuous communication and regular meetings with 
the working group, frontline staff, and statistical team is 
crucial in this continuous process of evaluation. Feed- 
backs and interim reporting help to identify deficiencies 
and direction of change at an earlier phase of the whole 
process of evaluation. Arrangement of site visit to the 
involved outpatient clinic and discussion with the front- 
line staff in addition to observation can help cross vali- 
date the accuracy of the data. 

Stakeholder endorsement and validation 

In a programme of this scale, small variations in 
programme protocol are inevitable between sites. Every 
clinic needs to be able to have the flexibility to introduce 
their own refinements tailored specifically to accommo- 
date the needs of their patient population. Other factors 
such as changes in availability of staff, space, and fluc- 
tuations in patient load can also contribute to variations 
in protocol delivery. To accommodate these variations, 
it is essential to verify with all participating clinics that 
the same operation skeleton is maintained and modifica- 
tions from the protocol be minimised and maintained 
within an acceptable limits in order to totally reflect the 
performance and impact of the RAMP. 

Furthermore, site visits to the clinics are an effective 
way to identify any potential on-site problems encoun- 
tered in the actual implementation of the programme 
and suggest possible solutions. 



Development of the evaluation framework 

The indicators and benchmark standards of care for the 
evaluation framework were derived from literature re- 
view and international and local management guidelines. 
However, research-based evidence is not always available 
for every criterion. In these circumstances, a comprom- 
ise between best practice and local practice needs to be 
agreed upon with both policy decision makers and cli- 
nical healthcare providers. Furthermore, clear operation 
definitions are required to describe what is being mea- 
sured, for example, explicit definitions for referral rates, 
enrolment rates and attendance rates need to be agreed 
to ensure that the information collected is accurate and 
interpretable. 

Defining the target standards for each criterion of care 
is also essential and requires a fine balance between 
standards which are ideal and standards which are fea- 
sible. While the standard of care should be set appro- 
priately high to allow room for improvement, it must 
still be realistic for use in real clinical settings. 

Operation definitions 

In order to ensure the reliability of the responses of the 
Structure of Care Questionnaires, and to define clearly 
what exact information is needed and what exact data 
should be retrieved, operation definitions are set and 
supplemented to the evaluation reference framework so 
that the interpretation of the framework is standardized. 
Clear articulation, field testing, and modification of 
poorly understood items are required to avoid misinter- 
pretation and assure accurate data collection, and it can 
be accomplished with the interim evaluation. 

Monitoring and ensuring data quality 

Anonymised data on the process of care, clinical out- 
comes, and service utilization outcomes is extracted 
from the Hospital Authority's information system. How- 
ever, there may be substantial amount of missing data on 
clinical outcomes of care particularly for non-participants. 
It is because while one of the advantages of having RAMP 
is to ensure regular checking of patients' clinical para- 
meters like HbAlc, urine albumin-to-creatinine ratio, etc., 
patients who have not taken in RAMP (non-participants) 
are not secured to have these clinical outcomes checked 
regularly and their clinical data may not be as comprehen- 
sive as that for patients who have enrolled into RAMP 
(RAMP participants). Moreover, some information such 
as the duration of having DM of both RAMP participants 
and non-participants are self-reported which significantly 
influences the information accuracy. Therefore, every 
piece of information collected should be interpreted care- 
fully in an attempt to truly reflect the performance and 
impact of the programme. To address these problems, it is 
important to discuss with the health policy decision 
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makers, coordinators, and clinical staff whether it is fea- 
sible to collect the valid information as much as possible 
and the acceptable sampling period should be made. 
Thereupon, the operation definition for the information 
needed should be concluded (Additional file 1). For 
example, for the reading of HbAlc at baseline, the 
last available reading prior to the date of RAMP enrol- 
ment rather than those after should be adopted. The 
last HbAlc reading available in 9-15 months after RAMP 
enrolment should be used as the post- 12 months reading. 

Conclusion 

With the expected increasing numbers of patients with 
chronic diseases like DM, a well-organized and compre- 
hensive multi-disciplinary approach to care will be the 
preferred model of healthcare delivery. Formal eva- 
luation studies are needed to provide evidence that these 
models help to enhance the quality of care offered and 
received by the patients. If proven effective, similar multi- 
disciplinary models can be developed for management of 
patients with other chronic diseases like hypertension, 
chronic obstructive sleep apnoea, or osteoporosis at the 
primary care level. Studies looking to establish evidence 
for these models of care will be of considerable interest 
to health care planners, particularly when aging popula- 
tion is the global trend. 

Additional files 



Additional file 1: Quality of Care Evaluation Framework. 
Additional file 2: Structure of care questionnaire. 
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